A Fault Localization Prioritization: Comparing Information Theoretic and Coverage Based Approaches

Fault localization can be used to prioritize the statements of a program according to the likelihood that each contains a fault that causes a known failure [Jones et al. 2002; Renieres and Reiss 2003]. Regression test prioritization is used to order statements according to early achievement of some testing goal such as coverage [Elbaum et al. 2000; Li et al. 2007; Rothermel et al. 2001; Yoo and Harman 2012]. Regression testing and fault localization are naturally complementary: we test to see if change has introduced a fault and, if we discover that it has, we switch from regression testing to fault localization as a first step towards fault fixing. Having fixed the fault we switch back to regression testing and so the cycle continues. This ‘test–find–fix’ cycle is familiar to many software engineers. Complete execution of all test cases in a regression test suite can be slow; execution times of up to seven weeks have been reported in the literature [Rothermel et al. 2001]. For larger test suites, it may not be realistic to expect the engineer to wait for all test cases to be executed before they can start work on fault fixing. Also, it may not be worth executing all test cases, once the first failure has been detected. In many scenarios, as soon as the first failure is detected, the engineer will want to switch from testing to fixing. Even if the engineer does not wish to make this switch at the first failure, there may be, at some subsequent test execution, a failure for which the engineer will want to switch from testing to fixing: for example, if the engineer expects that the failure is important, or requires immediate action. Also, if the engineer expects that the fault which causes the currently detected failure will also cause many cascaded downstream failures, then it would be wasteful to have to wait for regression testing to continue, flagging up all such cascaded errors in the process. There are many situations where this switch between testing and fixing may occur, particularly when many faults have been inserted. In such a situation, regression testing will be